This notebook will demonstrate the process of splitting a video into its component frames for an object recognition model.

Necessary libraries:

In [1]:
library(gdata)
library("jpeg")
library("imager")

gdata: Unable to locate valid perl interpreter
gdata: 
gdata: read.xls() will be unable to read Excel XLS and XLSX files
gdata: unless the 'perl=' argument is used to specify the location of a
gdata: valid perl intrpreter.
gdata: 
gdata: (To avoid display of this message in the future, please ensure
gdata: perl is installed and available on the executable search path.)
gdata: Unable to load perl libaries needed by read.xls()
gdata: to support 'XLX' (Excel 97-2004) files.

gdata: Unable to load perl libaries needed by read.xls()
gdata: to support 'XLSX' (Excel 2007+) files.

gdata: Run the function 'installXLSXsupport()'
gdata: to automatically download and install the perl
gdata: libaries needed to support Excel XLS and XLSX formats.

Attaching package: 'gdata'

The following object is masked from 'package:stats':

    nobs

The following object is masked from 'package:utils':

    object.size

The following object is masked from 'package:base':

    startsWith

Loading required package: magrittr

Attaching package: 'imager'

The following object is masked from 'package:magrittr':

    add

The following objects are masked from 'package:stats':

    convolve, spectrum

The following object is masked from 'package:graphics':

    frame

The following object is masked from 'package:base':

    save.image

With the videos captured, we use the following functions to convert them into images:

In [2]:
getFileNameAndExtension <- function(file){ 
    ex <- strsplit(basename(file), split="\\.")
    ex<-unlist(ex)
    return(ex)
}
In [3]:
convertVideoAsImages<-function(ffmpeg,videoFileName,fps,filePrefix,fileSuffix)
{
 nameAndExtension<-getFileNameAndExtension(videoFileName)
 utils::browseURL(videoFileName)
 dir.create(nameAndExtension[[1]])
 command<-sprintf("%s -i %s -vf fps=%d ./%s/%s%%d%s",ffmpeg,videoFileName,fps,nameAndExtension[[1]],filePrefix,fileSuffix)
 command
 system(command)
}

Now we input our file name and settings and convert the video into frames:

In [4]:
firstVideoFileName<-'marker.mov'
ffmpeg<-'c:/ffmpeg/bin/ffmpeg'
outFilePrefix='out'
outFileSuffix='.jpg'
fps=10
convertVideoAsImages(ffmpeg,firstVideoFileName,fps,outFilePrefix,outFileSuffix)
0

Here is an example frame from the video to ddemonstrate that the process was successfully performed:

In [5]:
nameAndExtension<-getFileNameAndExtension(firstVideoFileName)
fnames <- paste0(nameAndExtension[[1]],"/",outFilePrefix, 1, outFileSuffix)
jj <- readJPEG(fnames,native=TRUE)
plot(0:1,0:1,type="n",ann=FALSE,axes=FALSE)
rasterImage(jj,0,0,1,1)

We repeat the process for a second video:

In [6]:
secondVideoFileName<-'bottle.mov'
convertVideoAsImages(ffmpeg,secondVideoFileName,fps,outFilePrefix,outFileSuffix)
0
In [7]:
nameAndExtension<-getFileNameAndExtension(secondVideoFileName)
fnames <- paste0(nameAndExtension[[1]],"/",outFilePrefix, 1, outFileSuffix)
jj <- readJPEG(fnames,native=TRUE)
plot(0:1,0:1,type="n",ann=FALSE,axes=FALSE)
rasterImage(jj,0,0,1,1)

We use the following code to reduce the image size for classification, reducing the image to grayscale for simplicity:

In [8]:
compressRatio=40
nameAndExtension<-getFileNameAndExtension(firstVideoFileName)
fnames <- paste0(nameAndExtension[[1]],"/",outFilePrefix, 100, outFileSuffix)
sampleImage <- load.image(fnames)
plot(sampleImage)
In [9]:
thmb <- resize(sampleImage,round(width(sampleImage)/compressRatio),round(height(sampleImage)/compressRatio))
gs<-grayscale(thmb)
plot(gs)
In [10]:
gs.matrix<-as.matrix(gs)
dim(gs.matrix)
  1. 27
  2. 48
In [11]:
plot(thmb,main="Thumbnail")

We use the following function to save the reduced images in their own folder:

In [12]:
reduceImagesfromFolder<-function(reductionRatio,thumbNailFolderPrefix,imageFolder,prefix)
{
imageNames<-paste0(prefix,"*.*")
files<-list.files(path=imageFolder,pattern=imageNames,all.files=T,full.name=T,no..=T)
list_of_images=lapply(files,load.image)
n.images<-length(list_of_images)
outFolderName<-sprintf("%s%s",thumbNailFolderPrefix,imageFolder)
outFileName<-sprintf("%s%s",thumbNailFolderPrefix,files[[1]])
if(!dir.exists(outFolderName))
{
  dir.create(path=outFolderName)  
}
for(ii in 1:n.images)
{
  outFileName<-sprintf("%s%s",thumbNailFolderPrefix,files[[ii]])
  thmb <- resize(list_of_images[[ii]],round(width(list_of_images[[ii]])/reductionRatio),round(height(list_of_images[[ii]])/reductionRatio))
  thmb<-grayscale(thmb)
  imager::save.image(thmb,outFileName)
}
}

We call the function on the first image set and plot an example:

In [13]:
thumbNailFolder<-"Small"
nameAndExtension<-getFileNameAndExtension(firstVideoFileName)
reduceImagesfromFolder(compressRatio,thumbNailFolder,nameAndExtension[[1]],outFilePrefix)
markerex<-load.image("smallmarker/out1.jpg")
plot(markerex)

We call the function on the second image set and plot an example:

In [14]:
nameAndExtension<-getFileNameAndExtension(secondVideoFileName)
reduceImagesfromFolder(compressRatio,thumbNailFolder,nameAndExtension[[1]],outFilePrefix)
bottlex<-load.image("smallbottle/out1.jpg")
plot(bottlex)
In [ ]: